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Abstract 

In evolutionary optimization, it is important to understand how fast evolutionary algorithms converge to the optimum per 
generation, or their convergence rates. This paper proposes a new measure of the convergence rate, called the average convergence 
rate. It is a normalized geometric mean of the reduction ratio of the fitness difference per generation. The calculation of the 
average convergence rate is very simple and it is applicable for most evolutionary algorithms on both continuous and discrete 
optimization. A theoretical study of the average convergence rate is conducted for discrete optimization. Lower bounds on the 
average convergence rate are derived. The limit of the average convergence rate is analyzed and then the asymptotic average 
convergence rate is proposed. 


Index Terms 

evolutionary algorithms, evolutionary optimization, convergence rate, Markov chain, matrix analysis 


I. Introduction 

Evolutionary algorithms (EAs) belong to iterative methods. As iterative methods, a fundamental question is their convergence 
rates; how fast does an EA converge to the optimum per generation? According to m, existing results on the convergence 
rate of genetic algorithms can be classihed into two categories. The first category is related to the eigenvalues of the transition 
matrix associated with an EA. A lower bound of convergence rate is derived in El for simple genetic algorithms by analyzing 
eigenvalues of the transition matrix. Then the work is extended in || 3 ] and it is found that the convergence rate is determined 
by the second largest eigenvalue of the transition matrix. The other category is based on Doeblin’s condition. The upper bound 
on the convergence rate is derived using Deoblin’s condition in a. As to continuous optimization, the local convergence 
rate of EAs on the sphere function, quadratic convex functions and convex objective functions are discussed in a-Q. The 
research of the convergence rate covers various types of EAs such as isotropic algorithms 18 ], gene expression programming 
0, multiobjective optimization EAs Qol. The relationship between the convergence rate and population size is investigated 

in im, ina. 

The convergence rate in previous studies Q-a is based on Markov chain theory. Suppose that an EA is modeled by a 
hnite Markov chain with a transition matrix P, in which a state is a population ifOl . Let pt be the probability distribution of 
the fth generation population on a population space, tt an invariant probability distribution of P. Then p* is called convergent 
to 7r if limt^oo || pt — tt ||= 0 where || • || is a norm; and the convergence rate refers to the order of how fast pt converges 
to 7r 0. The goal is to obtain a bound e{t) such that || pt — tt ||< e{t). But to obtain a closed form of e(<) often is difficult 
in both theory and practice. 

The current paper aims to seek a convergence rate satisfying two requirements; it is easy to calculate the convergence rate 
in practice while it is possible to make a rigorous analysis in theory. Inspired from conventional iterative methods llT 4 l . a new 
measure of the convergence rate, called the average convergence rate, is presented. The paper is organized as follows; Section lU 
dehnes the average convergence rate. Section|In]establishes lower bounds on the average convergence rate. SectionlTVl discusses 
the connections between the average convergence rate and other performance measures. Section m introduces an alternative 
dehnition of the average convergence rate if the optimal fitness value is unknown. Section |Vl] concludes the paper. 

II. Definition and Calculation 

Consider the problem of minimizing (or maximizing) a function f{x). An EA for solving the problem is regarded as an 
iterative procedure (Algorithm [T]i; initially construct a population of solutions $0; then generate a sequence of populations $1, 
$2, $3 and so on. This procedure is repeated until a stopping criterion is satisfied. An archive is used for recording the best 
found solution. 

The fitness of population <I>t is defined by the best htness value among its individuals, denoted by Since /($*) is 

a random variable, we consider its mean value ft ;= E[/($t)]. Let /opt denote the optimal htness. The fitness difference 

This work was supported by EPSRC under Grant No. EP/I009809/1 (He), National Science Foundation of Guangdong Province under Grant No. 
S2013010014108 and Shenzhen Scientific Research Project under Grant No. JCYJ 20130401095559825 (Lin). 

Jun He is with Department of Computer Science, Aberystwyth University, Aberystwyth SY23 3DB, U.K. e-mail: jqh@aber.ac.uk. 

Guangming Li is with Shenzhen Institute of Information Technology, China, e-mail: lingm@sziit.com.cn. 


2 


Algorithm 1 An EA with an archive 
1 : initialize a population of solutions <i)o and set f ■<— 0 ; 

2: an archive records the best solution in $o; 

3: while the archive doesn’t include an optimal solution do 
4: generate a new population of solutions $t+i; 

5: update the archive if a better solution is generated; 

6: t i — f -f 1; 

7: end while 


between /opt and ft is |/opt — ft\- The convergence rate for one generation is 


/opt ft 
/opt ft—1 


( 1 ) 


Since |/opt — ft\ ~ |/opt — ft-i\, calculating the above ratio is unstable in practice. Thus a new average convergence rate for 
EAs is proposed in the current paper. 

Definition 1: Given an initial population $ 0 ^ the average (geometric) convergence rate of an EA for t generations is 


Rit I $o) := 1 - 


/opt fl 

/opt /o 


/opt ft 
/opt ft — 1 



/opt ft 

/opt /o 


1/t 


( 2 ) 


If /o = /opt, let R(t I $o) = 1- For the sake of simplicity, R{t) is short for R{t \ <i)o). 

The rate represents a normalized geometric mean of the reduction ratio of the fitness difference per generation. The larger 
the convergence rate, the faster the convergence. The rate takes the maximal value of 1 at ft = /opt- 

Inspired from conventional iterative methods M Definition 3.1], the average (logarithmic) convergence rate is defined as 
follows: 




/opt ft 

/opt /o 


(3) 


Eormula (O is not adopted since its value is +00 at ft = /opt- But in most cases, average geometric and logarithmic 
convergence rates are almost the same. Let at := |/opt — /t|/|/opt — /t-i|- Usually ctt ~ 1 and (ai ■ ■ -at)^!^ ~ 1 , then 
- log(ai - - - atf^^ ^ (1 - (ai - - - Ri 1 . 

In practice, the average convergence rate is calculated as follows: given f(x) with /opt known in advance, 

1) Run an EA for T times (T ^ 1). 

2) Then calculate the mean fitness value ft as follows. 


1 

T 



(4) 


where denotes the fitness f{^t) at the feth run. The law of large numbers guarantees (IH approximating to the 

mean fitness value ft = E(/(<l>t)) when T tends towards + 00 . 

3) Einally, calculate R(t) according to formula (|2|l. 

The calculation is applicable for most EAs on both continuous and discrete optimization. We take an example to illustrate 
the average convergence rate. Consider the problem of minimizing Ackley’s function: 


n n 

f{x) = — 20 exp{—0.2[y^(xi + e)^/n] 2 } — exp[^ cos(27ra;i + 27re)/n] + 20 + e, 

i=l i=l 


(5) 


where Xi € [—32 — e, 32 — e], i = 1, • • • ,n. The optimum is (—e, —e, - - -) and /opt = 0. We compare the Multi-grid EA 
(MEA) iflSi with the East Evolutionary Programming (EEP) ifTh) under the same experiment setting (where n is 30 and 
population size is 100). Run the two EAs for 1500 generations and 100 times. Calculate ft according to (0 and then R(t) 
according to (|2|i. Fig. [T] illustrates the convergence rates of MEA and EEP. 

The average convergence rate is different from the progress rate such as |/t — /opt| or logarithmic rate log \ft — /opt| used 
in El. The progress rate measures the fitness change; but the convergence rate measures the rate of the fitness change. We 
demonstrate this difference by an example. Let g(x) = 10Qf(x). In terms of \ ft — /opt|, the progress rate on g(x) is 100 times 
that on f(x). In terms of log \ ft — /opt|, the progress rate on g(x) is 1-1-2/ log \ ft — /opti times that on f(x). However, the 
average convergence rate is the same on both f(x) and g(x). 
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Fig. 1. A compai'ison of the average convergence rates R{t) of MEA and FEP on Ackley’s function. 


III. Analysis for Discrete Optimization 

Looking at Fig. [T] again, two questions may be raised; what is the lower bound or upper bound on R{t)l Does R{t) 
converge or not? For discrete optimization, a theoretical answer is provided to these questions in this section. For continuous 
optimization, its analysis is left for future research. 

In the rest of the paper, we analyze EAs for discrete optimization and assume that their genetic operators do not change 
with time. Such an EA can be modeled by a homogeneous Markov chain US with transition probabilities Pr(X, E) := 
Pr($(_|_i = F I = X),X,Y G S, where populations X,Y denote states of and S denotes the set of populations (called 
the population space). Let P denote the transition matrix with entries Pr(X, F). 

A population is called optimal if it includes an optimal solution; otherwise called non-optimal. Let So-pt denote the set of 
optimal populations, and ^non = S \ ^opt- Because of the stopping criterion, the optimal set is always absorbing. 


Pr($t +1 e S'non I G Sopt) = 0. 


(6) 


Transition matrix P can be split into four parts: 


*^opt ‘^non 

^ ^opt / A O \ 

^ ^non 'v B Q ) 


(7) 


where A is a submatrix representing probability transitions among optimal states; O a submatrix for probability transitions 
from optimal states to non-optimal ones, of which all entries take the value of zero; B a submatrix denoting probability 
transitions from non-optimal states to optimal ones; and Q a submatrix for probability transitions among non-optimal states. 

Since is a random variable, we investigate the probability distribution of <I>t instead of itself. Let qt{X) denote the 
probability of at a non-optimal state X, qt{X) Pr($t = X). Let vector {Xi,X 2 , • • •) represent all non-optimal states 
and vector denote the probability distribution of in the non-optimal set, where q* := {qt{Xi),qt{X 2 ),-■ ■)"'"■ Here 
notation q is a column vector and q^ the row column with the transpose operation. For the initial probability distribution, 
qo > 0 where 0 = (0,0, • • ■ )^. Only when the initial population is chosen from the optimal set, qg = 0. 

Consider probability transitions among non-optimal states only, which can be represented by matrix iteration 


qf = qf-iQ = qi^Q*- 


( 8 ) 


Definition 2: An EA is called convergent if limt^+oo qt = 0 for any qg or limt^+oo Q* = O. It is equivalent to saying 
that the probability of finding an optimal solution is 1 as f tends towards -foo. 

The mean fitness value ft is given as follows: 

/t:=E[/($t)]= ^/(A)Pr(<i>* = A). (9) 

xes 

Then it follows 

fopt-ft= Y. {f{X)-fopt)qt{X). (10) 


Let vector f := {f{Xi), /(A 2 ), • • • )^ denote the fitness values of all non-optimal populations (Xi,X2, ■ ■ ■)■ Then (fTOl i can 
be rewritten in a vector form 


/opt - /t = qf ■ (/opti - f), 
where • denotes the vector product and 1 = (1,1, • • • )^. 


( 11 ) 
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For a vector v, denote 


|:= |v^ • (/optl - f)|. 


Since || v ||= 0 iff v = 0; || av ||= |a| || v || and || vi + V 2 ||>|| Vi 
M, let II M II be the induced matrix norm, given by 

v^M II 


( 12 ) 

V 2 II, thus II V II is a vector norm. For a matrix 


M 11= sup 


: V ^ 0 


(13) 


Using the above Markov chain model, we are able to estimate lower bounds on the average convergence rate. 

Theorem 1: Let Q be the transition submatrix associated with a convergent EA. For any qo ^ 0, 

1) The average convergence rate for t iterations is lower-bounded by 

R{t) >1- IIQ* ir/‘. (14) 

2) The limit of the average convergence rate for t generations is lower-bounded by 

lim R{t) > 1 — p(Q), (15) 

f+OO 

where p(Q) is the spectral radius (i.e., the supremum among the absolute values of all eigenvalues of Q). 

3) Under random initialization (that is, Pr($o = X) > 0 for any X G Snon or qo > 0), it holds 

lim R{t) = l — p{Q). (16) 

4) Under particular initialization (that is, se{3 qo = v/|v| where v is an eigenvector corresponding to the eigenvalue p(Q) 

with V > 0 but V 0. The existence of such a v is given in the proof), it holds for all t > 1, 


R{t) = l-p{Q). 


Proof: 1) From qj = qJfQ*, we have 

I/opt - ft\ 


qt 




I/opt - /o| 


T 

qo 


q^ll - 


q?’ nil Q‘ 

II q^ II 


=11 Q‘ 


Hence 


1 - 


/opt ft 


/opt — fo 


1/t 


> 1- II Q‘ 


(17) 


(18) 


(19) 


( 20 ) 


which proves the first conclusion. 

2) According to Gelfand’s spectral radius formula ifTSl p.619], we get 

lim ||Q*|r/‘=p(Q). 

The second conclusion follows by combining (l20l i with (fT4l i. 

3) Since Q > 0, according to the extension of Perron-Frobenius’ theorems to non-negative matrices M PP- 670], p(Q) is 
an eigenvalue of Q. There exists an eigenvector v corresponding to p(Q) such that v > 0 but v ^ 0. In particular, 

p(Q)v'^ = v^Q. (21) 

Let max(v) denote the maximum value of the entries in vector v. Due to random initialization, qg > 0. Let min(qo) denote 
the minimum value of the entries in vector qg. Set 


From (I 2 TI) . we get 


_ inin(qo) 
max(v) 

p(Q)u^ = u^Q. 


( 22 ) 


(23) 


Thus vector u is an eigenvector of /o(Q). 

Let w = qg — u. Then from (l22l l. we know w > 0. Since qg = u -f w, w > 0 and Q > 0, we deduce that 


qi = q^ Q* = (u -f w)^ Q‘ > Q* = p(Q)‘u^ . 


(24) 


*For vector v = (fi ,V 2 , - ■ ■), denote |v| := '^2i l^^l' 
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It follows that 


/opt ft 


qf ■ (/optl - f) 

> 

p(Q)‘u'^ ■ (/optl - f) 

/opt /o 


/opt /o 


qo ■ (/optl - f) 


/opt - ft 


/opt /o 


1/t 


> p(Q) 


(/opti - f) 


Qo ■ (/optl - f) 


i/t 


Since both |u^ • (/optl — f)| and • (/optl — f)| are independent of t, we let t —>■ +cxd and get 


lim 


U 


(/optl - f) 


Qo ■ (/optl - f) 


1/t 


= 1 , 


then we get 


lim 

f + OO 


/opt ft 

/opt /o 


lit 

> p(Q)- 


(25) 

(26) 


(27) 


(28) 


lim R{t) = 1 — lim 

t—¥-\-oo t—^-\-CO 


/opt ft 

/opt — /o 


lit 

< l-p(Q). 


(29) 


The third conclusion follows by combining ( |29] ) with ( fTSl ). 

4) Set qo = v/ ut where v is given in Step 3. Then qo is an eigenvector corresponding to the eigenvalue p(Q) such 
that p(Q)qj^ = q^Q- From ([8]i: qf = q?liQ, we get 


/opt - ft ^ q/ ■ (/optl - f) ^ p(Q)*qo ■ (/optl - f) 
/opt ~ /o /opt ~ /o qj^ ■ (/optl — f) 

Thus we have for any t > 1 


/opt ft 

/opt /o 


1/t 

= piQ), 


(30) 


then R{t) = 1 — p(Q) which gives the fourth conclusion. ■ 

The above theorem provides lower bounds on the average convergence rate. Furthermore, it reveals that R{t) converges to 
1 — /o(Q) under random initialization and R{t) = 1 — p(Q) for any f > 1 under particular initialization. Similar to conventional 
iterative methods M pp. 73], we call 1 —p(Q) the asymptotic average convergence rate of an EA, denoted by Rex,. According 
to (fThl l. its value can be approximately calculated as follows; under random initialization, R{t) approximates to 1 — p(Q) if 
t is sufficiently large. Note that this definition is different from another asymptotic convergence rate, given by — log p(Q) 
in ESI. In most cases, the two rates are almost the same since usually p(Q) ~ 1 and then — logp(Q) (1 — p(Q)) ~ 1- Since 
1 — p(Q) is independent of t and initialization, hence using asymptotic average convergence rate is convenient for comparing 
two EAs, for example, to analyze mixed strategy EAs im. 


IV. Connections 

The average convergence rate is different from other performance measures of EAs: the expected hitting time is the total 
number of generations for obtaining an optimal solution 113; and fixed budget analysis focuses on the performance of EAs 
within fixed budget computation 1^ . However, there are some interesting connections between them. 

There exists a link between the asymptotic average convergence rate and the hitting time. Let m{X) be the expected 
number of generations for a convergent EA to hit S'opt when starting from state X (called the expected hitting time). Denote 
m := {m{Xi),m{X 2 ), • • • )^ where (Xi,X 2 , • • •) represent all non-optimal states. 

Theorem 2: Let Q be the transition submatrix associated with a convergent EA. Then l/i?oo is not more than || m ||oo:= 
max{m{X);X G S'non}- 

Proof: According to the fundamental matrix theorem IItTI Theorem 11.5], m = (I — Q)~^l, where I is the unit matrix. 
Then 

II m lU =11 (I - Q)-'i ||oo=|| (I - Q)-' ||oo> p((l - Q)-') = (1 - p(Q))-\ (31) 

where the last equality takes use of a fact; (1 — p(Q))”^ is an eigenvalue and spectral radius of (I — Q)“^. ■ 

The above theorem shows that l/i?oo is a lower bound on the expected hitting time. 

Eollowing Theorem [T] a straightforward connection can be established between the spectral radius p(Q) and the progress 
rate |/opt - ft\- 

Corollary 1: Let Q be the transition submatrix associated with a convergent EA. 
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1) Under random initialization (that is qo > 0), it holds 


1- I/opt ft\ _ , 

t4+cx.p(Q)‘|/„pt-/o| - • 


(32) 


2) Under particular initialization (that is, set qg = v/|v| where v is an eigenvector corresponding to the eigenvalue p(Q) 
with V > 0 but V ^ 0), it holds for all t > 1, 


I/opt -/tl 

p(Q)‘|/opt-/o| 


= 1 . 


(33) 


The exponential decay, p(Q)* *|/opt — fo\, provides a theoretical prediction for the trend of |/opt — ft\- The corollary conhrms 
that the spectral radius p(Q) plays an important role on the convergence rate Q. 

We explain the theoretical results by a simple example. Consider a (1+1) EA for maximizing the OneMax function |x| 
where x = (si, • • • , s„) S {0,1, }". 


Algorithm 2 A (1+1) elitits EA 

Onebit Mutation: choose a bit of <l>t (one individual) uniformly at random and flip it. Let rlrt denote the child. 
Elitist Selection: if /(T't) > /($*), then let <l>t+i ^ 'i't; otherwise <l>t+i ^ $t. 


Denote subset Sk '■= {x ; |x| = n — k} where fc = 0, • 
Sk) = ^ and Pr($t+i G S'?,; | e Sfe) = 1 - Writing 


0 


2 

n 



■ ■ ,n. Transition probabilities satisfy that Pr($t+i G Sk-i 
them in matrix P (where submatrix Q in the bold font); 

■••0 0 0 \ 

• 0 0 0 

• 0 0 0 


G 


(34) 


0 0 0 ■■•^1-^0 
\0 0 0 ••• 0 1 0/ 

The spectral radius p(Q) = 1 — 4 and the asymptotic average convergence rate Roo = 4. Notice that l/i?oo(= n) is less 
than the expected hitting time (= n(l + | + • • • + 4)). 

In the OneMax function, set n = 10, and then p(Q) = 0.9 and R^o = 0.1. Choose <i)o uniformly at random, run the (1+1) 
EA for 50 generations and 2000 times, and then calculate ft according to (|4]l and R{t) according to formula (|2]i. Since $0 is 
chosen uniformly at random, /g « 5. Eig. |2] demonstrates that R{t) approximates 0.1 (= Roo)- Eig.|3] shows that the theoretical 
exponential decay, p(Q)*|/opt — /o|, and the computational progress rate, |/opt — ft\, coincide perfectly. 



Fig. 2. R{t) approximates 0.1 for the (1+1) EA on the OneMax function with n = 10. 


V. Alternative Rate 

So far the calculation of the average convergence rate needs the information about /opt- However this requirement is very 
strong. Here we introduce an alternative average convergence rate without knowing /opt, which is given as below. 


R^{t) := 1 - 


ft+st ft 


l/St 


ft - ft-St 

where St is an appropriate time interval. Its value relies on an EA and a problem. 


( 35 ) 
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Fig. 3. A comparison of the theoretical prediction p(Q)*|/opt — /o| and the computational result |/opt — ft \ for the (1+1) EA on the OneMax function 
with n = 10, fo = 5, /opt = 10 and p(Q) = 0.9. 


For the (1+1) EA on the OneMax function with n = 10, we set St = 10. Choose $o uniformly at random, run the (1+1) 
EA for 60 generations and 2000 times, and then calculate ft according to (|4|i and R^{t) according to formula dlST l. Due to 
St = 10, R^{t) has no value for / < 10 and / > 50 according to formula (I351 l. Eig.a demonstrates that R^{t) approximates 
0.1(= 1 — p(Q)). But the calculation of R^{t) is not as stable as that of R{t) in practice. 



Fig. 4. R^(t) approximates 0.1 for the (1+1) EA on the OneMax function with n = 10. 


The above average convergence rate converges to 1 — p(Q) but under stronger conditions than that in Theorem [T] 

Theorem 3: Let Q be the transition submatrix associated with a convergent EA. 

1) Under particular initialization (that is, set qo = v/|v| where v is an eigenvector corresponding to the eigenvalue p(Q) 
with V > 0 but V ^ 0), it holds for all t > 1, 

/?*(/) = l-p(Q). (36) 

2) Under random initialization (that is qo > 0), choose an apprOTriate St such that g := (I — Q'*‘)(/optl — f) > 0 for a 
maximization problem (or g < 0 for a minimization problem^ If Q is positiv^E then it holds 

lim Ri{t) = 1-piQ). (37) 


Proof: Erom (l8]i: q^ = q^iQ and (fTTTi . we get 

ft+St ~ ft = ft+6t — /opt + /opt ~ ft = (It ' (/optl ~ f ) ~ flt+it ■ (/optl ~ f) 

= qf ■ (/optl - f) - qf Q^*(/optl - f) = qf ■ g. (38) 

1) Since qo is an eigenvector corresponding to the eigenvalue p(Q) such that p(Q)qQ = qjf^Q. Erom (l3^ and ([^: 
qf = q?liQ, we get 


ft 

l/St 

qf g 

IjSt 

qo Q*g 

l/St 

piQY 

T 

„ qo ■ g 

ft ft—St 


(list ■ g 


qo Q‘“'^‘g 



r s 

qo g 


P(Q)- 


(39) 


^It is always true for a large time interval 6t since lim 5 t_^_|_oo(I — = I and /optl — f > 0 for a maximization problem (or /optl — f < 0 for a 

minimization problem). 

^The condition of positive Q could be relaxed to non-negative Q if taking a similai' argument to the extension of Perron-Frobenius’ theorems to non-negative 
matrices E] pp. 670]. 

















Then R^{t) = 1 — p(Q) which gives the first conclusion. 
2) Without loss of the generality, consider g > 0. Since 


let 


ft+5t ft _ Qt ■ § 

ft ~ ft—St ^'t—st' S 


A( = min 


hl- 


<5tJ* 


^I-stQ'^s 

qf-it-g ’ 




= max 

i 


hl-sth 


where [v]i represents the ith entry in vector v. 

According to Collatz formula ll22ll ll^ Theorem 2], 

lim A( = lim A* 


p(Q'*). 


(40) 


(41) 


(42) 


Hence for any [g]i > 0, it holds 


lim min 

t—>-+oo i 


[qf_5tQ'*‘]i[g]* 

l^I-StUsh 


piQ'% 


lim max 

t —>'+oo i 


[ciLstQ%[sh 

hl-stUsh 


Using min{^,fa} < 


Ql~l~Q2 

&1+&2 


< max{fi, fa}, we get 


lim 

t—f + OO 


E.K-stQ%is]. 

EM-sMsh 


P(Q'‘)- 


Equivalently 


lim 

t—¥-\-00 


qr^tQ^*g 

qf-at • g 


p(Q'‘)- 


Then 


lim 

t—¥-\-00 


ft 

ft ft—St 


l/6t 


= p(Q). 


p(Q'‘)- 


(43) 


(44) 


(45) 


(46) 


Finally it comes to the second conclusion. ■ 

The theorem shows that the average convergence rate R^{t) plays the same role as R{t). But the calculation of R^{t) is not 
as stable as that of R{t) in practice. 


VI. Conclusions 

This paper proposes a new convergence rate of EAs, called the average (geometric) convergence rate. The rate represents 
a normalized geometric mean of the reduction ratio of the fitness difference per generation. The calculation of the average 
convergence rate is simple and easy to implement on most EAs in practice. Since the rate is normalized, it is convenient to 
compare different EAs on optimization problems. 

For discrete optimization, lower bounds on the average convergence rate of EAs have been established. It is proven that 
under random initialization, the average convergence rate R{t) for t generations converges to a limit, called the asymptotic 
average convergence rate; and under particular initialization, R{t) equals to the asymptotic average convergence rate for any 
t > 1. 

The analysis of EAs for continuous optimization is different from that for discrete optimization. In continuous optimization, 
an EA is modeled by a Markov chain on a general state space, rather than a finite Markov chain. So a different theoretical 
analysis is needed, rather than matrix analysis used in the current paper. This topic is left for future research. 

References 

[1] L. Ming, Y. Wang, and Y.-M. Cheung, “On convergence rate of a class of genetic algorithms,” in Proceedings of 2006 World Automation Congress. 
IEEE, 2006, pp. 1-6. 

[2] J. Suzuki, “A Markov chain analysis on simple genetic algorithms,” IEEE Transactions on Systems, Man and Cybernetics, vol. 25, no. 4, pp. 655-659, 
1995. 

[3] E. Schmitt and E. Rothlauf, “On the importance of the second largest eigenvalue on the convergence rate of genetic algorithms,” in Proceedings of 2001 
Genetic and Evolutionary Computation Conference, H. Beyer, E. Cantu-Paz, D. Goldberg, Parmee, L. Spector, and D. Whitley, Eds. Morgan Kaufmann 
Publishers, 2001, pp. 559-564. 

[4] J. He and L. Kang, “On the convergence rate of genetic algorithms,” Theoretical Computer Science, vol. 229, no. 1-2, pp. 23—39, 1999. 

[5] G. Rudolph, “Local convergence rates of simple evolutionary algorithms with Cauchy mutations,” IEEE Transactions on Evolutionary Computation, 
vol. 1, no. 4, pp. 249-258, 1997. 

[6] -, “Convergence rates of evolutionary algorithms for a class of convex objective functions,” Control and Cybernetics, vol. 26, pp. 375-390, 1997. 















9 


[7] -, “Convergence rates of evolutionary algorithms for quadratic convex functions with rank-deficient hessian,” in Adaptive and Natural Computing 

Algorithms. Springer, 2013, pp. 151-160. 

[8] O. Teytaud, S. Geliy, and J. Mary, “On the ultimate convergence rates for isotropic algorithms and the best choices among various forms of isotropy,” 
in Parallel Problem Solving from Nature (PPSN IX). Springer, 2006, pp. 32^1. 

[9] X. Du and L. Ding, “About the convergence rates of a class of gene expression programming,” Science China Information Sciences, vol. 53, no. 4, pp. 
715-728, 2010. 

[10] N. Beume, M. Laumanns, and G. Rudolph, “Convergence rates of SMS-EMOA on continuous bi-objective problem classes,” in Proceedings of the Ilth 
Workshop on Foundations of Genetic Algorithms. ACM, 2011, pp. 243-252. 

[11] M. Jebalia and A. Auger, “Log-linear convergence of the scale-invariant {pjp.yj, A)-ES and optimal p for intermediate recombination for large population 
sizes,” in Parallel Problem Solving from Nature (PPSN XI). Springer, 2010, pp. 52-62. 

[12] F. Teytaud and O. Teytaud, “Convergence rates of evolutionary algorithms and parallel evolutionary algorithms,” in Theory and Principled Methods for 
the Design of Metaheuristics. Springer, 2014, pp. 25-39. 

[13] J. He and X. Yao, “Towards an analytic framework for analysing the computation time of evolutionary algorithms,” Artificial Intelligence, vol. 145, no. 
1-2, pp. 59-97, 2003. 

[14] R. Varga, Matrix Iterative Analysis. Springer, 2009. 

[15] J. He and L. Kang, “A mixed strategy of combining evolutionary algorithms with multigrid methods,” International Journal of Computer Mathematics, 
vol. 86, no. 5, pp. 837-849, 2009. 

[16] X. Yao, Y. Liu, and G. Lin, “Evolutionary programming made faster,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 2, pp. 82-102, 1999. 

[17] R. Salomon, “Evolutionary algorithms and gradient search: similarities and differences,” IEEE Transactions on Evolutionary Computation, vol. 2, no. 2, 
pp. 45-55, 1998. 

[18] C. Meyer, Matrix Analysis and Applied Linear Algebra. SIAM, 2000. 

[19] J. He, F. He, and H. Dong, “Pure strategy or mixed strategy?” in Evolutionary Computation in Combinatorial Optimization, J.-K. Hao and M. Middendorf, 
Eds. Springer, 2012, pp. 218-229. 

[20] T. Jansen and C. Zarges, “Performance analysis of randomised search heuristics operating with a fixed budget,” Theoretical Computer Science, vol. 545, 
pp. 39-58, 2014. 

[21] C. Grinstead and J. Snell, Introduction to Probability. American Mathematical Society, 1997. 

[22] L. Collatz, “EinschlieBungssatze fiir charakteristische zahlen von matrizen,” Mathematische Zeitschrift, vol. 48, no. 1, pp. 221-226, 1942. 

[23] R. J. Wood and M. O’Neill, “Finding the spectral radius of a large sparse non-negative matrix,” ANZIAM Journal, vol. 48, pp. 330-345, 2007. 



